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Abstract 

Background: Public and patient involvement in the different stages of the health technology assessment (HTA) 
process is increasingly encouraged. The selection of topics for assessment, which includes identifying and 
prioritizing HTA questions, is a constant challenge for HTA agencies because the number of technologies requiring 
an assessment exceeds the resources available. Public and patient involvement in these early stages of HTA could 
make assessments more relevant and acceptable to them. Involving them in the development of the assessment 
plan is also crucial to optimize their influence and impact on HTA research. The project objectives are: 1) setting up 
interventions to promote patient participation in three stages of the HTA process: identification of HTA topics, 
prioritization, and development of the assessment plan of the topic prioritized; and 2) assessing the impact of 
patient participation on the relevance of the topics suggested, the prioritization process, and the assessment plan 
from the point of view of patients and other groups involved in HTA. 

Methods: Patients and their representatives living in the catchment area of the HTA Roundtable of Universite 
Laval's Integrated University Health Network (covering six health regions of the Province of Quebec, Canada) will be 
involved in the following HTA activities: 1) identification of potential HTA topics in the field of cancer; 2) revision of 
vignettes developed to inform the prioritization of topics; 3) participation in deliberation sessions for prioritizing 
HTA topics; and 4) development of the assessment plan of the topic prioritized. The research team will coordinate 
the implementation of these activities and will evaluate the process and outcomes of patient involvement through 
semi-structured interviews with representatives of the different stakeholder groups, structured observations, and 
document analysis, mainly involving the comparison of votes and topics suggested by various stakeholder groups. 

Discussion: This project is designed as an integrated approach to knowledge translation and will be conducted 
through a close collaboration between researchers and knowledge users at all stages of the project. In response to 
the needs expressed by HTA producers, the knowledge produced will be directly useful in guiding practices 
regarding patient involvement in the early phases of HTA. 

Keywords: Public and patient participation, Research priorities, Health technology assessment, Public involvement, 
Knowledge translation 



Background 

Public and patient involvement is recognized as an ef- 
fective approach to improving the relevance of research, 
increasing the value of scientific research without com- 
promising its rigour, facilitating the implementation of 
innovations, and increasing the external validity of 
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results [1,2]. In health technology assessment (HTA), 
public and patient involvement is also considered as a 
priority [3-8]. As direct beneficiaries of health services, 
patients have a comprehensive knowledge of the impact 
and the effects of a treatment or a technology on their 
condition and on different aspects of their life [5,9]. 

Several authors have noted that decisions for selecting 
technologies to assess should be more focused on the 
values and needs of patients and the public [4,5,10]. The 
selection of HTA topics which includes identifying tech- 
nologies for assessment and prioritizing them, is part of 
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the mandate of HTA units and agencies. Because the 
number of technologies requiring an assessment signifi- 
cantly exceeds the resources available, all HTA organiza- 
tions face a problem of prioritization [11]. Some studies 
have shown that patients' priorities differ from those of 
researchers and clinicians [2,12-15]. In the field of cancer, 
Corner and Wright [12] have demonstrated that although 
the biological aspects and those related to treatment were 
considered important by patients, they rated the manage- 
ment of practical, social and emotional issues as a higher 
priority. The results echoed those of an earlier study con- 
ducted by the U.S. National Cancer Institute in 1997 [13] 
showing that although patients and their families were in 
favour of cancer research, they felt that research often 
more clearly served the interests of clinicians and re- 
searchers than the priorities of the people directly affected 
[13]. Including patients' experiential knowledge in the se- 
lection of HTA topics makes it possible to counterbalance 
the potential biases brought by scientists or health profes- 
sionals [16,17]. By involving patients and the public in the 
identification and prioritization of HTA topics, assess- 
ments are more likely to be relevant to them and adapted 
to their needs [10,18]. 

A review of the different roles for patients and the 
public in HTA shows that many agencies worldwide give 
a role to patients and/ or to the public in the identifica- 
tion of assessment topics [10]. Most of the time, a down- 
loadable electronic form is available to members of the 
general public on the websites of HTA organizations, 
thus enabling them to suggest assessment topics [10]. A 
more proactive approach is also used by the program of 
the National Institute for Health Research (NIHR) in the 
United Kingdom [19-21]. This organization collaborates 
with community groups and the James Lind Alliance, a 
non-profit initiative funded by the NIHR, whose purpose 
is to ensure that publicly funded research corresponds to 
what matters to both patients and clinicians [22] . Commu- 
nity groups are invited to submit suggestions for research 
topics based on their experiences with particular health 
conditions. In the United States, the Patient-Centered 
Outcomes Research Institute also seeks questions from 
patients through a website, social media and engagement 
activities [23]. 

Regarding the prioritization of HTA topics, several agen- 
cies across the world, including the Canadian Agency for 
Drugs and Technologies in Health [24], are using similar 
criteria: the burden of disease associated with the technol- 
ogy, the potential clinical impact, the potential economic 
repercussions, the impact on the funding budget of the 
technology, the availability of evidence on the technology, 
and the presence of alternatives [11]. Other context-specific 
criteria are also considered, such as the expected level of 
public interest, the potential controversy of the topic, the 
accessibility and reimbursement of the technology, and the 



variation in the rates of use [24]. However, there is little 
knowledge available on how these criteria are actually used 
to guide decisions in the selection of HTA topics [11]. 

Although several researchers consider that the prioritization 
stage in HTA is value-laden and that a greater involve- 
ment of stakeholders (including patients) would be im- 
portant [10,25], patients are mostly excluded from this 
stage in HTA [10], with some exceptions. One of these is 
the HTA program of NIHR in the United Kingdom (UK), 
which has established a whole infrastructure to support 
public involvement in the identification and prioritization 
of assessment topics [26,27] . Under this program, the pub- 
lic contributes to prioritization in two ways: as reviewers 
of vignettes that serve as the basis of discussion of the 
prioritization committees and as members of the commit- 
tees themselves [10,21]. Another experience of public in- 
volvement in prioritization of HTA topics is that of the 
Centers for Medicare and Medicaid Services in the USA, 
which involve citizens in prioritization by offering them 
the opportunity to comment upon potential issues that 
are posted on their website [10]. 

The development of the assessment plan related to the 
topic selected for HTA is also a stage at which it is par- 
ticularly relevant to involve patients [28-30]. This stage, 
which includes the identification of relevant issues, di- 
mensions, measures and indicators related to the topic 
selected, appears crucial to involve patients because it 
enables an optimizing of their influence and impact on 
the entire research initiative [4,5,29,30]. User involve- 
ment at this stage can favour the relevance of research 
questions to patients' needs and ensure that the indicators 
and measurement instruments actually reflect the dimen- 
sions they want to include [14,20,28,30]. 

According to our conceptual framework of patient in- 
volvement in HTA, three approaches concerning the 
mechanisms of patient involvement could be distinguished: 
active participation, consultation, and communication/ 
information (see Figure 1). Each of these approaches is 
characterized by the level of patient involvement (participa- 
tion as full partners, consultants, or recipients of informa- 
tion) and by a set of corresponding techniques or methods. 

Mitton and collaborators [31] have reviewed the vari- 
ous techniques or methods of public involvement used 
in the prioritization and allocation of resources. Despite 
the lack of available evidence, their review suggests that 
using deliberative methods may be associated with more 
satisfactory results than using other methods that are 
more consultative and characterized by a lower level of 
public involvement [31]. Methods that involve face-to- 
face interaction between participants from the public 
and decision makers also generate greater participant 
satisfaction, both in terms of process and results [31]. 
Furthermore, a study by Abelson [32] demonstrates that 
the views of members of the public concerning priorities 



Gagnon et al. BMC Health Services Research 2014, 14:273 
http://www.biomedcentral.eom/1 472-6963/1 4/273 



Page 3 of 7 



HTA process 




> 



Selection of 
evaluation topics 
V 



Evaluation 



Diffusion 



Submetting 
requests 



Prioritizing 
requests 



Developing an 
assessment 
plan 



Collecting 
evidence 
(Literature 
or new 
data) 



ST 



Analysing 

and 
syntheising 
results 





Discussing 

report and 
recomman 
dations 

\ A. 



Diffusing 
report 



Level 1 -Information 

Mechanisms are put in 
place to inform and raise 
awareness among patients 

f Level 2- Consultation "V 

Mechanisms are put in 

place to collect 
information or opinions 
from patients 



r 



Level 3 - Participation 

Mechanisms are put in 
place to actively involve 
patients as partners 

\ 



f > 

Identify potential 
HTA topics in the 
field of cancer 






f Revise 
vignettes 

Participe in 
priorisation 
L committees A 


Participate in 

working 
group for the 
assessment 
plan 



Figure 1 Interventions to promote patient involvement in the early stages of the HTA process. Legend: Levels of patient involvement 
according to the stages of the HTA process. 



differ systematically when they are given the opportunity 
to discuss the topics. According to this study, delibera- 
tive methods could facilitate a more informed choice in 
the prioritization of topics than methods such as ques- 
tionnaires or individual interviews, where the opinion of 
the public participants is requested only once and with- 
out first being discussed. 

Goal and setting of the project 

This project builds upon a previous project entitled In- 
volving patients in HTA activities at local level: a study 
protocol based on the collaboration between researchers 
and knowledge users [33] which allowed the involvement 
of patients in the HTA process of assessing alternatives to 
restraint and seclusion for hospitalized or institutionalized 
adults. Based on the results of this experiment, this new 
project will involve patients in HTA stages that have not 
been considered in our previous work, namely the identifi- 
cation and prioritization of HTA topics, and the develop- 
ment of the assessment plan of the prioritized topic. 

HTA producers need to find effective ways to incorp- 
orate the perspective of patients in HTA structures and 



activities [3,25]. This project has been developed in col- 
laboration with the HTA Roundtable of the Integrated 
University Health Network (IUHN) of Universite Laval 
(Quebec, Canada) that was set up in order to support 
healthcare organizations throughout its territory in HTA 
and knowledge transfer. This HTA roundtable recently ini- 
tiated a consultation among its member organizations (in 
a catchment area covering the whole of eastern Quebec) 
whose results showed that assessments could better meet 
local and regional needs. In Quebec, patients' experience 
regarding their choice of treatments, technologies, and 
service modalities can differ depending on the region. Re- 
gions with a university hospital have easier access to spe- 
cialized and subspecialized care, and new technologies are 
usually introduced earlier in these regions. 

In continuity with our previous work, and to improve 
the consideration of regional and local needs, the HTA 
roundtable of this IUHN collaborated with researchers 
from the project in order to involve patients with other 
stakeholders from the six health regions of its catchment 
area in the identification and prioritization of HTA topics 
related to cancer. The theme of cancer was chosen by 
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knowledge users of the team because it represents the 
leading cause of death in Canada [34], and the entire 
problem area raises a very broad array of questions and is- 
sues that encompass the full range of services and health 
policies, ranging from primary prevention, screening of tar- 
get populations, access to healthcare services, and treat- 
ment and expensive drugs, to palliative and end of life care. 
Moreover, cancer represents a field in which new tech- 
nologies, including drugs, evolve quickly, and for which 
indications of current technologies also expand rapidly, 
raising several relevant HTA questions. 

The main study objectives are: 1) to set up interventions 
promoting patient involvement in these regions in the se- 
lection of HTA topics (identification and prioritization) 
and in the development of the assessment plan of the 
chosen subject, and 2) to assess the impacts of these inter- 
ventions from the point of view of patients and other 
groups involved in HTA. In this project, the term patient 
refers to a person affected by a given technology (currently 
or in the past), but also includes patient representatives 
such as community groups. 

Methods 

Patients will be involved in four specific activities of the 
HTA process (see Figure 1): 1) identification of HTA 
topics; 2) revision of vignettes that will be developed to 
inform the prioritization of the topics; 3) prioritization 
of HTA topics; and 4) development of the assessment 
plan. Different mechanisms of involvement will be used 
according to the level of involvement (consultation and 
direct participation) and the type of activity (topic sugges- 
tion in a web form, vignette review, deliberative meetings 
with other stakeholders to prioritize topics, HTA working 
group). The research team will observe and evaluate the 
process and outcomes associated with patient involvement 
in all these activities. A participatory evaluation approach 
will be used [35-38], involving stakeholders at all stages of 
the research process (planning, design, data collection and 
analysis, identification of outcomes, conclusions, recom- 
mendations, and dissemination of results). 

Identification of HTA topics 

We will undertake consultations among various health- 
care organizations and patient and community groups 
from the IUHN territory in order to identify potential 
topics for HTA in the field of cancer. With the collabor- 
ation of the CEOs of the regional health and social services 
agencies, the regional coordinator of oncology services, 
and community organizations in cancer collaborating to 
the project, we will contact healthcare professionals, man- 
agers and community organizations to invite them to sug- 
gest topics by way of an online or paper form. They will 
also be invited to diffuse information about the project in 
their community. We will provide instructions and specific 



examples on the suggestion form. Participants will have to 
indicate whether they are patients (current or past), close 
relatives of patients, professionals, or healthcare managers 
in order to allow for a consideration of the specificities of 
each of these groups. 

Filtration of topics and preparation of vignettes 

Following the example of the NIHR HTA program in 
the United Kingdom [20], we will then undertake a fil- 
tration of the suggested topics. The topics will be exam- 
ined by trained staff from the HTA unit of the Quebec 
University Hospital Centre (QUHC) whose members will 
check their relevance for an HTA program and whether 
they have already been covered by a previous HTA pro- 
ject. Moreover, we will set up a central prioritization com- 
mittee assembling representatives of different categories 
of stakeholders (managers, healthcare professionals and 
patients) from the various regions covered by the project. 
This committee will have to validate recommendations 
made by the HTA unit. The six regions covered by the 
study will be divided into three groups: university, periph- 
eral and intermediary, and remote [39] . As a patient repre- 
sentative will be recruited in each group of regions, at 
least three patient representatives will participate in the 
prioritization meeting in order to validate the final list 
of topics. 

Based on the UK experience [20,40], we will then pre- 
pare short vignettes (less than one page) for each topic 
in order to provide preliminary information on the re- 
search question (technology, group of patients affected, 
and context) and the potential effectiveness of the technol- 
ogy. These vignettes will be sent to experts from relevant 
fields for comments. At least one patient representative, se- 
lected from those referred by patient organizations, will be 
asked to review the vignettes. Reviewers will comment 
upon the relevance and importance of the research and the 
knowledge that could be produced [19]. 

Prioritization of HTA topics 

We will organize deliberation sessions in each region cat- 
egory (university, peripheral or intermediary, and remote) 
with groups of approximately nine people - including 
healthcare professionals, decision makers and patients - 
to prioritize HTA topics. Based on the vignettes produced 
at the previous stage, participants will be invited to discuss 
the technologies presented. Two members of the project 
team, including an HTA producer, will facilitate the delib- 
eration sessions. Finally, each participant will vote indi- 
vidually, and, in secret, will rate the proposed topics and 
classify them in priority order. The votes of the partici- 
pants will be compiled according to the category to which 
they belong [41]. 

The central prioritization committee set up during the 
filtration stage will meet again to establish, through a 
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deliberative method, a list of five topics in order of prior- 
ity, as a means of identifying one priority topic that will be 
evaluated as well as other important topics that could be 
assessed in the future. The final decision as regards 
accepting the proposed HTA topic will revert back to the 
HTA Roundtable of Universite Laval's Integrated Univer- 
sity Health Network, which could mandate the HTA unit 
of the QUHC and its partners to conduct the assessment. 

Following the recommendations of the UK HTA pro- 
gram [19], a third of participants in the deliberation and 
prioritization sessions must be patient representatives. 
The recruitment of these representatives will begin with 
a description of the work and the profile of the potential 
participant, based on criteria developed by the NHS in 
the UK [20,40]. The collaboration of cancer- related com- 
munity organizations in this project will make it possible 
to identify relevant candidates. To maximize the contri- 
bution of patient representatives, we will try to recruit 
individuals who are connected to peer networks and are 
thus able to suggest a wide corpus of patients. We will 
also try to respect balance regarding gender and age in 
each of the groups. 

The evaluation of public involvement in HTA prioritization 
in the UK has demonstrated the importance of providing 
dedicated staff and regular feedback to support this in- 
volvement [19,26,27,40,42]. Therefore, patients will receive 
training in HTA and evidence-informed healthcare before 
the prioritization meeting [20]. A member of the research 
team will also provide ongoing support for patient repre- 
sentatives throughout the process. 

Development of the assessment plan 

For the last stage, a working group consisting of various 
stakeholders, including patients, will be responsible for 
the refining the topic selected and developing the assess- 
ment plan. They will specify the research question, the 
dimensions to be evaluated, and the strategies to imple- 
ment in order to conduct this HTA, notably regarding 
patient involvement in the next stages of the process. 
The HTA process will then continue under the auspices 
of the HTA unit of the QUHC, so this study will stop 
following the development of the assessment plan, but 
our previous collaborative projects with this HTA unit 
will have laid the groundwork for pursuing patient in- 
volvement at the next stages of the project. 

Evaluation of processes and outcomes associated with 
patient involvement 

The research team will observe and evaluate the process 
and outcomes associated with patient involvement through- 
out the project using a participatory evaluation approach 
[35-38] . Our team has already used this approach success- 
fully in a previous project [33]. A workshop will be con- 
ducted at the beginning of the project with knowledge 



users of the team to reach a consensus on the evaluation 
objectives and questions. Data collection methods, indica- 
tors of effects, outcomes and a data-analysis plan will also 
be discussed, as well as the roles and responsibilities of 
each team member. Such an approach is recognized as be- 
ing the most effective for knowledge translation [43,44]. It 
helps improve the outcomes of an intervention because 
the results are consistent with practical information needs 
of knowledge users and speak to the challenges they have 
to face [45] . It also allows stakeholders to recognize them- 
selves in the results and recommendations made, which 
facilitates their use in decision making [45]. 

To assess interventions, we will consider the criteria re- 
lated to the processes and those associated with the results 
or impacts. We will partly apply the model proposed by 
Rowe [46], which identifies nine criteria for assessing pa- 
tient involvement: those related to the construction and 
implementation of the process, i.e. 1) representativeness, 
2) independence, 3) early involvement, 4) influence or im- 
pact on decision making, 5) transparency; and those re- 
lated to the process, including 6) accessibility of resources 
to fulfill its role, 7) task definition, 8) structured decision 
making and 9) cost-effectiveness. However, a subsequent 
analysis of these criteria by Rowe and collaborators [47] 
showed that although they have a certain validity, they 
are not exhaustive, nor are they necessarily appropri- 
ate for all involvement activities or all contexts. The 
choice of outcome indicators, methods and analysis strat- 
egies will be based on discussions with researchers and de- 
cision makers, an approach recognized as being the most 
effective for knowledge translation [43,44]. 

The evaluation of all interventions will be based mainly 
on a qualitative approach, through in-depth interviews 
and observations. The interviews will explore patients' 
contributions and their influence on the process and re- 
sults, as perceived by various stakeholders. Patient satis- 
faction and perceptions regarding the final list of topics 
and criteria that justified their choice will be of particular 
interest to us. The interviews will also allow us to collect 
suggestions to improve the process. The deliberation ses- 
sions will be observed to analyze the course of discussions 
and decision making. Document analysis will allow com- 
paring the research priorities identified by patient repre- 
sentatives, healthcare professionals, and managers. 

The evaluation could contribute to improving current 
knowledge on the impact of patient involvement in the 
selection of HTA topics and on the factors influencing 
this involvement. We will also pay particular attention 
to the possibility of transposing the interventions imple- 
mented to other HTA contexts. The proposed approach 
is important in order to complete the knowledge transla- 
tion cycle [48] since the knowledge produced through 
this project will be useful for the development of future 
interventions to support patient involvement in HTA. 
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Ethical considerations 

Ethics approval for the project has been received from 
the Research Ethics Board of CHU de Quebec (approved 
on September 19, 2013; ethics number C13-08-1761). 
Interviewees will be asked to complete a consent form 
presenting the research objectives and information about 
research implications. 

Discussion 

This project will help setting up interventions to pro- 
mote patient participation in three stages of the HTA 
process, and assessing the impacts of these interventions 
on the relevance of the topic prioritized and the assess- 
ment plan from the perspectives of patients and other 
groups involved in the project. This study will comple- 
ment our previous project, which enabled the involvement 
of patients in a specific HTA on alternative measures to 
restraint and seclusion [33]. In that particular project, in- 
volvement of patients in the stages of identification, 
prioritization, and development of the assessment plan 
was not possible due to the timing of the research. Based 
on the positive results of this experience, the present pro- 
ject will make it possible to involve patients in the early 
stages of HTA that have not been considered in our previ- 
ous work. 

Despite various initiatives to involve patients in the 
HTA stages, assessments of the impact of these initia- 
tives are rather sparse and anecdotal [10]. Evidence on 
the effectiveness of public participation (including pa- 
tients) in HTA is only beginning to emerge [14,40]. The 
project will therefore allow us not only to apply know- 
ledge from other studies in the Canadian context, but 
also to evaluate proposed interventions, and thus will 
provide essential knowledge on the effectiveness of the 
involvement strategies put in place by paying particular 
attention to the involvement process, including its im- 
plementation and follow-up, while considering the views 
of different stakeholders. If we do not want this involve- 
ment to remain anecdotal, it is important for knowledge 
users to rely on evidence to develop strategies for patient 
involvement in HTA decision making [49]. 

Knowledge translation activities 

In this project, knowledge translation is adapted to the 
local context and to knowledge users' needs because of 
the collaborative approach adopted. One of the strategies 
will consist in presenting progress reports and results 
periodically to all stakeholders involved in the project. 
We will also communicate the results of each phase to 
stakeholders from the regions concerned through the 
participation of researchers from the team in the HTA 
Roundtable of the IUHN. Targeted presentations will 
provide a forum for discussing the research process, data 
analysis, and interpretation of results. Presentations will 



also be made to provide information about the process 
of the project and present its results to a wider audience 
(Ministry of Health and Social Services, Quebec National 
Institute of Public Health, Health and Welfare Commission, 
as well as relevant patient organizations). This will facili- 
tate a consideration of the potential for dissemination 
and the applicability of the interventions to other or- 
ganizations interested in HTA. These presentations will 
be integrated into the regular training activities of the 
organizations concerned. A brief questionnaire will be 
distributed to participants after each meeting to assess 
the relevance of these meetings, the understanding of 
key messages, and the intention to apply them in 
their practice. 

The research team will work with the HTA Roundtable 
of the IUHN to share and disseminate lessons learned with 
the Quebec community of practice in HTA, and other pro- 
vincial, national and international HTA organizations. To 
adapt the dissemination of results to different audiences, 
representatives of all groups involved in the project will 
participate in the dissemination process. Collaborations 
with patient groups and community organizations will 
also facilitate the sharing of results directly with patients. 

Abbreviations 

HTA: Health technology assessment; IUHN-LU: Integrated University Health 
Network (IUHN) of Laval University (LU); QUHC: Quebec University Hospital 
Centre; CIHR: Canadian Institutes of Health Research. 

Competing interests 

The authors declare that they have no competing interests. 
Authors' contributions 

MPG, BC, FL, JG and MD conceived and designed the study and drafted the 
manuscript. DL, MC and MR participated in designing the study and revised 
the manuscript. MTP helped to draft the manuscript. All authors read and 
approved the final manuscript. 

Authors' information 

MPG is Tier 2 Canada Research Chairholder in Technologies and Practices in 
Health. FL is Tier 2 Canada Research Chairholder in Implementation of 
Shared Decision Making in Primary Care. 

Acknowledgements 

This study is funded by a Knowledge to Action grant from the Canadian 
Institutes of Health Research (CIHR; grant #201210KAL-289993-KAL-CFBA-1 1 1 141). 

Author details 

Research Centre of the CHU de Quebec, Hopital Saint-Francois d'Assise, 10 
rue de I'Espinay, D6-726, Quebec City, QC G1L 3L5, Canada. 2 Faculty of 
Nursing, Universite Laval, Quebec City, QC, Canada. 3 National institute of 
public health, Quebec City, QC, Canada. 4 CHU de Quebec, Quebec City, QC, 
Canada, department of Family Medicine, Universite Laval, Quebec City, QC, 
Canada. 

Received: 21 November 2013 Accepted: 16 June 2014 
Published: 20 June 2014 

References 

1. Wallerstein N, Duran B: Community-based participatory research 
contributions to intervention research: the intersection of science 
and practice to improve health equity. Am J Public Health 2010, 
100(Suppl 1):S40-S46. 



Gagnon et al. BMC Health Services Research 2014, 14:273 
http://www.biomedcentral.eom/1 472-6963/1 4/273 



Page 7 of 7 



2. Wright D, Corner J, Hopkinson J, Foster C: Listening to the views of people 
affected by cancer about cancer research: an example of participatory 
research in setting the cancer research agenda. Health Expect 2006, 
9(1):3-12. 

3. Bridges JF, Jones C: Patient-based health technology assessment: A vision 

of the future. Int J Technol Assess Health Care 2007, 23(1)30-35. 

4. Coulter A: Perspectives on health technology assessment: response from 
the patient's perspective. Int J Technol Assess Health Care 2004, 20(1):92-96. 

5. Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, Single A: 
Patients' perspectives in health technology assessment: a route to 
robust evidence and fair deliberation. Int J Technol Assess Health Care 
2010, 26(3):334-340. 

6. Hailey D, HTA Initiative # 21: Consumer involvement in health technology 
assessment. HTA Initiative # 21. Edmonton: Alberta Heritage Foundation for 
Medical Research; 2005. Available from: http://www.ihe.ca/documents/HTA- 
FR21.pdf. 

7. Lehoux P, Williams-Jones B: Mapping the integration of social and ethical 
issues in health technology assessment. Int J Technol Assess Health Care 
2007, 23(01):9-16. 

8. Pivik J, Rode E, Ward C: A consumer involvement model for health 
technology assessment in Canada. Health Policy 2004, 69(2):253-268. 

9. Health Equality Europe: Understanding Health Technology Assessement 
(HTA). In HEA Guide to HTA for Patients. ; 2008:48. 

10. Menon D, Stafinski T: Role of patient and public participation in health 
technology assessment and coverage decisions. Expert Rev Pharmacoecon 
Outcomes Res 2011, 11 (1):75-89. 

11. Noorani HZ, Husereau DR, Boudreau R, Skidmore B: Priority setting for 
health technology assessments: a systematic review of current practical 
approaches. Int J Technol Assess Health Care 2007, 23(3)310-315. 

12. Corner J, Wright D, Hopkinson J, Gunaratnam Y, McDonald JW, Foster C: 
The research priorities of patients attending UK cancer treatment 
centres: findings from a modified nominal group study. Br J Cancer 2007, 
96(6):875-881. 

13. Jenks S: The public applauds cancer research but not how research 
priorities are set. J Natl Cancer Inst 1 997, 89(5)350-351 . 

14. Staley K: Exploring impact: Public involvement in NHS, public health and social 
care research. Eastleigh: INVOLVE; 2009. 

15. Tallon D, Chard J, Dieppe P: Relation between agendas of the 
research community and the research consumer. Lancet 2000, 
355(9220):2037-2040. 

16. Caron-Flinterman JF, Broerse JE, Teerling J, Bunders JF: Patients' priorities 
concerning health research: the case of asthma and COPD research in 
the Netherlands. Health Expect 2005, 8(3):253-263. 

17. Entwistle VA, Renfrew MJ, Yearley S, Forrester J, Lamont T: Lay perspectives: 
advantages for health research. BMJ 1998, 31 6(71 29):463-466. 

18. Brett J, Staniszewska S, Mockford C, Seers K, Herron-Marx S, Bayliss H: The 
PIRICOM StudyA systematic review of the conceptualisation, measurement, 
impact and outcomes of patients and public involvement in health and social 
care research. United Kingdom Clinical Resarch Collaboration: London; 2009. 

19. Moran R, Davidson P: An uneven spread: a review of public involvement 
in the National Institute of Health Research's Health Technology 
Assessment program. Int J Technol Assess Health Care 201 1, 27(4)343-347. 

20. Public involvement, [http://www.nets.nihr.ac.uk/ppi] 

21. NIHR Health Technology Assessment programme and National Institute for 
Health Research: Identifying and prioritising HTA research. In., vol. 
http://www.nets.nihr.ac.uk/identifying-research. Southampton, UK. 

22. The James Lind Alliance, http://www.lindalliance.org/index.asp. 

23. How We Select Research Topics, http://www.pcori.org/funding-opportunities/ 
funding-announcements/how-we-select-research-topics/. 

24. Husereau D, Boucher M, Noorani H: Priority setting for health technology 
assessment at CADTH. Int J Technol Assess Health Care 2010, 26(3)341-347. 

25. Abelson J, Giacomini M, Lehoux P, Gauvin FP: Bringing 'the public' into 
health technology assessment and coverage policy decisions: from 
principles to practice. Health Policy 2007, 82(1)37-50. 

26. Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, Gyte G, 
Oakley A, Stein K: Involving consumers in research and development 
agenda setting for the NHS: developing an evidence-based approach. 
Health Technol Assess 2004, 8(15):1-148. III-IV. 

27. Royle J, Oliver S: Consumer involvement in the health technology 
assessment program. Int J Technol Assess Health Care 2004, 20(4):493-497. 



28. Boote J, Baird W, Beecroft C: Public involvement at the design stage 
of primary health research: a narrative review of case examples. 

Health Policy 2010, 95(1 ):1 0-23. 

29. Gagnon M-P, Desmartis M, Gagnon J, St-Pierre M, Gauvin FP, Rhainds M, 
Lepage-Savary D, Coulombe M, Tantchou Dipankui M, Legare F: Introducing 
the patient's perspective in hospital health technology assessment 
(HTA): the views of HTA producers, hospital managers, and patients. 
Health Expect 201 2, doi:1 0.1 1 1 1/hex.1 201 0. 

30. Staniszewska S, Jones N, Newburn M, Marshall S: User involvement in 
the development of a research bid: barriers, enablers and impacts. 
Health Expect 2007, 10(2):1 73-183. 

31. Mitton C, Smith N, Peacock S, Evoy B, Abelson J: Public participation in 
health care priority setting: A scoping review. Health Policy 2009, 

91 (3):21 9-228. 

32. Abelson J, Eyles J, McLeod CB, Collins P, McMullan C, Forest PG: Does 
deliberation make a difference? Results from a citizens panel study of 
health goals priority setting. Health Policy 2003, 66(1 ):95- 106. 

33. Gagnon MP, Gagnon J, St-Pierre M, Gauvin FP, Piron F, Rhainds M, 
Coulombe M, Lepage-Savary D, Desmartis M, Tantchou Dipankui M, Legare 
F: Involving patients in HTA activities at local level: a study protocol 
based on the collaboration between researchers and knowledge users. 
BMC Health Serv Res 201 2, 1 2:1 4. 

34. Statistics Canada: Leading causes of death.Consulted on September 19, 
2012. 2009. http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/ 
hlth36a-eng.htm. 

35. Weiss C: Have We Learned Anything New about the Use of Evaluation? 

Am J Eval 1998, 19(1):21-33. 

36. Guba EG, Lincoln YS: Fourth Generation Evaluation. Newbury Park: Sage; 
1989. 

37. Patton M: Utilization-focused Evaluation. 4th edition. Thousand Oaks, CA: 
Sage; 2008. 

38. White MA, Verhoef MJ: Toward a patient-centered approach: incorporating 
principles of participatory action research into clinical studies. Integr Cancer 
Ther 2005, 4(1)21-24. 

39. Service du developpement de I'information: Les regions sociosanitaires 
quebecoises regroupees. Ministere de la Sante et des Services sociaux du 
Quebec; 2008. vol. www.msss.gouv.qc.ca. MSSS, SDI, fevrier 2008. 

40. Oliver S, Armes DG, Gyte G: Public involvement in setting a national 
research agenda: a mixed methods evaluation. Patient 2009, 2(3):1 79-1 90. 

41. Johanson R, Rigby C, Newburn M, Stewart M, Jones P: Suggestions in 
maternal and child health for the National Technology Assessment 
Programme: a consideration of consumer and professional priorities. 
J R Soc Promot Health 2002, 1 22(1 ):50-54. 

42. Oliver S, Armes D, Gyte G: Evaluation of public influence on the NHS 
Health Technology Assessment Programme. In Social Science Research Uni. 
University of London: Institute of Education; 2006. 

43. Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers' perceptions 
of their use of evidence: a systematic review. J Health Serv Res Policy 2002, 
7(4):239-244. 

44. Lomas J: Health services research. BMJ 2003, 327(7427):1 301 -1 302. 

45. Preskill H, Compton DW, Baizerman M, Smith IE: Integrating theory and 
practice. Conceptual frameworks of the CEFP. Cancer Pract 2001, 
9(Suppl 1):S17-S22. 

46. Rowe G, Frewer L: Public participation methods: a framework for 
evaluation. Sci Technol Hum Val 2000, 25(1)3-29. 

47. Rowe G, Horlick-Jones T, Walls J, Poortinga W, Pidgeon NF: Analysis of a 
normative framework for evaluating public engagement exercises: 
reliability, validity and limitations. Public Underst Sci 2008, 17(4):41 9-441. 

48. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson 
N: Lost in knowledge translation: time for a map? J Contin Educ Health 
Prof 2006, 26(1 ):1 3-24. 

49. Gagliardi AR, Lemieux-Charles L, Brown AD, Sullivan T, Goel V: Barriers 
to patient involvement in health service planning and evaluation: 
an exploratory study. Patient Educ Couns 2008, 70(2):234-241 . 



doi:1 0.1 1 86/1 472-6963-1 4-273 

Cite this article as: Gagnon et al:. Involving patient in the early stages of 
health technology assessment (HTA): a study protocol. BMC Health 
Services Research 2014 14:273. 



